System for co-ordination of logical sequence of instructions across electronic devices using visual programming and wireless communication

ABSTRACT

An orchestration engine provides a technical output across multiple programmable objects such as electronic devices, virtual objects and cloud based services in response to user specified logic. The orchestration engine may be deployed on a mobile computer, a tablet computer, a laptop computer, a desktop computer, a wired or wireless electronic device in the system or on a server computer connected via internet. The orchestration engine is capable of supporting extensibility in order to expand support for similar common interaction methods to newer electronic devices via a plug-in framework by specifying the communication protocol of the new element and its capabilities in a descriptive way via a markup language. The orchestration engine is provided along with a library of drag and drop Visual Programming Language steps required for providing executable computer program steps for specifying a user specified logic by computer language illiterate person.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/337,523, filed Mar. 28, 2019, which is incorporated herein in itsentirety.

FIELD OF INVENTION

This invention pertains to computer related inventions. Moreparticularly it relates to an Orchestration engine providing a technicaloutput. Still more particularly, this invention pertains to anOrchestration engine that orchestrates providing a technical output on auser specified logic across two or more than two programmable objectsselected from the group consisting of electronic devices, virtualobjects and cloud services. Still more specifically, the OrchestrationEngine can be used by a computer language illiterate user.

BACKGROUND OF THE INVENTION

Building technology enabled experiments and prototypes involve acombination of electronic devices (e.g. sensors, actuators), cloud basedservices, user interface and communication between all these components.This requires knowledge and skills in the fields of electronics, circuitdesign, computer programming, networking and communication technologies.These diverse set of skills and the complexity involved requires a teamof individuals with relevant skills along with significant amount oftime and resources. It is desirable that the efficiency of the processof building technology enabled experiments and prototypes be improved torequire minimal amount of skills, time and resources. This objective isnot achievable by currently known computer program products, systems andmethods in Information Technology. The object of this invention is toovercome this inability.

Additionally, objective of this invention is also to enable computerprogramming language illiterate person to create technology enabledexperiments and prototypes without requiring any training inconstructing computer readable programs, circuit design and networkingprotocols.

U.S. Pat. No. 9,465,588 has disclosed a user programmable toy systemconfigured for programming a toy comprising; a microcontroller; acomputer memory configured for storing digital data provided by asoftware for controlling and executing by said microcontroller; at leastone transducer to provide a response according to a signal provided bysaid microcontroller; at least one sensor for seeding a signal to saidmicrocontroller; an improved Easy Format software enabling said user tocompose a user program for directing said microcontroller to provide aresponse when a signal is received from said sensor and to energize saidtransducer according to the program flow of said user program; whereinsaid improved Easy Format software comprises additional features of anEasy Format software, which includes the steps of: (a) defining m inputchannels for said controller unit to receive external data or signals,wherein m is an integer equal or greater than one; (b) defining n outputchannels for said controller to transmit data or signals, wherein n isan integer equal or greater than one; (c) specify x configuration state,wherein each configuration state defines the configuration of at leastone input channel and/or one output channel and x is an integer equal orgreater than one; (d) specify y events to be executed by said controllerunit, wherein y is an integer equal or greater than one; (e) specify anevent to be executed when the signal or data received by an inputchannel satisfied a predefined qualifying condition and (f) enable anevent to switch from a current configuration state to anotherconfiguration state; said Easy Format software further comprises atleast one of the following steps or characteristics: (g) a step tospecify at least one of the events to comprise of p sub-events, whereinp is an integer equal or greater than two; (h) a step to specify anevent to be triggered by a timer or timing signal; (i) a step tostructure the programming specifications into at least two fields,wherein the first field specifies the characteristics of one or morestates, and the second field specifies one or more events/sub-events;(j) a step to organize at least a part of the data specified by saidsteps into a table format; (k) wherein said states and events are notnecessary to be listed in sequential relationship to each other; (l) astep to label part of a table format program for specifying the power updefault condition of said user programmable toy system; and wherein saidimproved Easy Format software is further configured to support at leastone of the following additional characteristics (1) to (3): (1) whereinsaid transducer is a sound transducer, and said improved Easy Formatsoftware is further configured to handle communication with an externalresource remote or separated from said programmable toy set, and tofetch data representing one or more sound files obtained from saidexternal resource and for said microcontroller to process said data andgenerate a sound with said sound transducer; (2) wherein said userprogrammable toy set comprises a master toy member and two or moreaccessory toy members; wherein each accessory toy member comprises anelement for said master toy member to identify an accessory toy member;said software program is further configured to enable said user programto direct said microcontroller to provide a response when a predefinedaccessory toy member is identified; and (3) wherein said userprogrammable toy set further comprising a real time clock generator,wherein said user program allows said user to define a specific time ofa day and to direct said microcontroller to provide a response when thetime of said real time clock matches with the specific time defined bysaid user; and wherein any of said additional characteristics arecompatible with said configuration states and/or events format for auser of said programmable toy system to compose an improved Easy Formatprogram; and wherein said improved Easy Format software is furtherconfigured to compile or translate a user composed program having any ofsaid characteristics into codes executable by said microcontroller.

U.S. Pat. No. 6,246,927 has disclosed a method of managing a pluralityof controllable objects in the storage and execution of instructionsrelated to a performance of n desired actions, the method comprising thesteps of: (a) advising said plurality of objects that a first selectedobject is about to store instructions related to a performance of afirst desired action; (b) storing, in said first object, saidinstructions related to said performance of said first desired action;(c) advising said plurality of objects that said storing of saidinstructions in said first selected object has been completed; (d)preparing said plurality of objects to receive instructions related to aperformance of a next desired action; (e) storing, in a next selectedobject, said instructions related to said performance of said nextdesired action; (f) advising said plurality of objects that said storingof said instructions in said next selected object has been completed;and (g) repeating steps (d) through (f) until instructions related tosaid performance of said n desired actions have been stored.Additionally, U.S. Pat. No. 6,246,927 has disclosed is a system claimbased essentially of same steps as mentioned above.

U.S. Pat. No. 6,246,927 has also disclosed a programmable object capableof intercommunicating with other programmable objects to perform aseries of desired actions, comprising: transmitting means fortransmitting signals to be received by said other objects; receivingmeans for receiving signals transmitted by said other objects; storagemeans for storing a set of instructions related to a performance by saidobject of a desired action of said series of desired actions; assigningmeans for assigning a unique event code to each set of instructionsstored so that each desired action is represented by said assigned eventcode, said event code being stored in said storage means; trackingmeans, in communication with said assigning means, for tracking saidunique event code so as to permit said object to track which desiredaction of said series of desired actions is stored in said object;commencement means, in communication with said transmitting means, forcausing said transmitting means to transmit a first advisory signalindicative of the commencement of the storing of said event-codedinstruction set in said storage means and to transmit a second advisorysignal indicative of the completion of the storing of said event-codedset of instructions; interpreting means, in communication with saidreceiving means, said assignment means and said tracking means, forinterpreting advisory signals received by said receiving means so as topermit said object to track whether said other objects have storedanother event-coded set of instructions and to track what event codeshave been assigned to those other objects, so that said assignment meanscan determine a next available event code for assignment; executingmeans for executing said event-coded set of instructions; and playbackmeans, in communication with said transmitting means, for causing saidtransmitting means to transmit a playback signal, said playback signalcausing said object and said other objects receiving said playbacksignal to execute all stored event-coded sets of instructions inevent-coded order, so as to cause the performance of all desired actionsin said series of actions.

Still further U.S. Pat. No. 6,246,927 has disclosed a method of andsystem of controlling a plurality of controllable, spontaneouslyprogrammable toys in the storage and execution of user inputinstructions related to a performance of a series of n desired actions,which is essentially intended to execute the steps recited the methodcomprising the steps recited above on the method of managing a pluralityof controllable objects in the storage and execution of instructionsrelated to a performance of n desired actions.

Further U.S. Pat. No. 6,246,927 has disclosed is a system for storingand performing a series of n desired actions in the order stored,comprising: a plurality of programmable objects, each object comprising:memory means for storing instructions related to a performance of adesired action of a series of n desired actions; tracking means fortracking said instructions that have been stored in said object; andperformance means, in communication with said tracking means, forexecuting said instructions so as to perform said desired action in theorder in which said instructions related to said desired action wasstored in said object.

U.S. Pat. No. 6,546,436 has disclosed a programmable toy interface forcontrolling a toy structure having at least one electrically-poweredoutput device adapted to communicate with a computer having aprogrammable toy control and an identification program, the interfacecomprising: a memory having an identification data portion that storesinterface identification data specific to the toy structure; a toycontroller that executes the toy control program; and a power supplythat supplies electrical power to the controller. U.S. Pat. No.6,546,436 also discloses a programmable toy entertainment system havingabove features and a first computer loaded with a control,program-development program which generates control and command signalsand interface identification data and the first toy structure beingelectrically connected to the interface and capable of producing, inresponse to at least one output signal from the interface, at least onecontrolled electric power output. U.S. Pat. No. 6,546,436 also providesfor a programmable toy interface for electrical connection with andcontrol of a toy structure having at least one electrically-poweredoutput device, and that is adapted to communicate with a computer havinga programmable toy control and identification program, wherein the saidinterface comprising a radio frequency modem adapted to receive commandand interface identification code signals from at least the computer andfor transmitting data to at least one of the computer and a secondinterface; a controller in communication with the memory and the modemfor executing the toy control program, and further adapted to processdata stored in the memory for transmission via the modem; and a powersupply that supplies electrical power to the controller and forselectively supplying electrical output power in the range of 2 to 9volts dc to the output device.

US 20170053550 discloses an education system comprising a plurality ofautomation devices, a tablet computer and a data service center forprogramming, wherein the tablet computer is used to integrate drag anddrop visual programming language and the plurality of automation devicesto turn them into robotics and computer language educational tool; thedata service center is used to provide data service support.

US 20170053550 further discloses that the tablet computer communicateswith the plurality of automation devices via wireless communication, thedata service center communicates with the tablet computer via internet;the plurality of automation devices are different toys, the data servicecenter is a server-based data service center or a cloud-based dataservice center and comprises at least a server; each automation devicecomprises a CPU, a wireless transceiver, many actuators, many auxiliaryhardwares, at least an indicator, many sensors and a IR emitter; thewireless transceiver, said many actuators, said many auxiliaryhardwares, the at least an indicator, said many sensors and the IRemitter are electrically connected with the CPU; the wirelesstransceiver is used to receive command from the tablet computer and sendsensor information to the tablet computer; the wireless transceiver is aWiFi transceiver or a Bluetooth transceiver; said many actuators areused to make movements of the automation devices and drive wheels of theautomation devices; said many auxiliary hardwares are used to controlthe IR emitter and the at least an indicator; said many sensors are usedto detect situation of the automation device and comprise a camera, anIR radar, a microphone, an ultrasound detector and a weight detector;the tablet computer comprises a software application that houses avisual programming language interpreter, a wireless transceiver and anumber of translators; wherein the software application can implement anew educational function in the automation devices; different softwareapplications are run on different tablet computers, different VPLprograms can be written on different software applications; the visualprogramming language interpreter can support at least a computerlanguage, a visual programming language program can be built in thetablet computer through visual-programming-language blocks; the visualprogramming language program can be integrated in the automation devicesto trigger meaningful actions of the automation devices; the translatorscan map the functionalities of the visual-programming-language blocks todifferent automation devices in different sets of instructions; theserver saves information of an educational function wherein theinformation comprises courses, lessons of each course, classes asinstances of lessons with date, time and venue; the lessons havestencils and notes; the classes save the user's homework or assignmentand teacher's grading and carry date and time data to indicate when alesson is being held for particular users; the visual programminglanguage program can be stored in the server and downloaded by the user;when the programs on a tablet computer is running, if the currentexecution statement is responsible to control an automation device tomove forward, the tablet computer will then send the move forwardcommand to the automation device immediately; the VPL programs can bedesigned to start execution upon a particular user event or hardwareevent detected by ultrasound sensor, IR received by the IR radar, aparticular pattern recognized by the camera which are triggered by anautomation device's sensors; each event is represented by one eventblock in the VPL programs. When an event is fired, the sequence ofblocks stringed after corresponding event block will start to execute.

Though the above existing systems provide different capabilities none ofthem provide computer programming language illiterate individual theabilities to experiment with or implement technology enabled ideas orconcepts that span electronic devices, cloud services, user interfacecomponents and virtual objects simultaneously. Further, by abovedescribed prior art; before executing the user specified logic onelectronic devices, the logic cannot be validated using mock objectscreated on the computing device (e.g. tablet or computer). A mock objectis a simulated object that mimics the behavior of an electronic devicein controlled ways. This limits the ability of the user to test andvalidate the logic in absence of the electronic devices. It is desirableto have such mock objects and support for testing user specified logic.

Another drawback is that in the prior art systems, the ability ofexecuting the user defined logic is implemented on either the electronicdevice or a computing device (e.g. computer, tablet). It is desirable toprovide the user the choice of selecting the mode of logic executionbest suited for the desired/contemporary application of the userspecified logic.

Further, none of the above prior art systems, methods or objects areenabled for computer language skill illiterate users with a capabilityto specify their logic to be orchestrated in form of inputs, outputs,triggers and actions between one electronic device or virtual object orcloud services and at least one more electronic device or virtual objector cloud service and validate by running such a program.

SUMMARY OF THE INVENTION

This invention embodies an Orchestration Engine for providing atechnical output across two or more than two programmable objectsselected from the group consisting of electronic devices, virtualobjects and cloud services in response to user specified logic. Theorchestration engine is deployed on a mobile computer or a tabletcomputer or a laptop computer or a desktop computer or a wired orwireless electronic device in the system or on a server computerconnected via internet. The said technical output comprises actionperformed in response to an event or a trigger generated by one or moreelectronic devices, virtual objects or cloud based services and acombination of different programming and logic constructs such asconditional statements, loops, variables. The orchestration engine isalso provided with an input of the user specified logic through: (i)Visual Programming Interface, or (ii) Interpreted Programming language;and (iii) the Orchestration Engine is capable of executing the sameusing a communication protocol for communication between two or more ofthe programmable objects selected from the group electronic devices,virtual objects and cloud based services. The orchestration engine hasone or more of following features: (i) the orchestration engine providesuser with means for speed setting and waits for user specified timebetween every step in the logic allowing user to analyze the sequence ofevent at a human perceivable speed, (ii) the orchestration engine iscapable of analyzing the logic specified by the user, compare it againsta database of logic specified by other users and come up withsuggestions and recommendations to the user about possible improvementsto the logic. The electronic device to which the technical output isprovided has one or more of following features: (a) is capable ofdetecting presence of other similar components in their vicinity usingbroadcast advertising on the wireless channel and by measuring thereceived signal strength of other wireless broadcasts received fromneighbouring devices, (b) is equipped with an ability to sensecapacitive touch, by using capacitive touch sensitive material inmanufacturing of the outer shell of the electronic devices, the toolthen, via the graphical user interface identifies/indicates whichelement the user has picked up, the Orchestration Engine then refers tothe database in cloud to identify which device is kept on the screen andthen displays appropriate help to the user.

The said event or a trigger generated by one or more electronic devicescomprises any one or an equivalent of following listed illustrativeevents or triggers, without limitation: (i) when presence of anotherobject in vicinity is sensed by a proximity sensor device, (ii) whenacceleration or movement in any direction is sensed by motion sensordevice, (iii) when an email is received with user specified content atuser specified account, (iv) when an image on screen of a mobile ortablet or laptop or desktop computer is clicked with mouse or touched onthe touchscreen by the use, (v) when variation in ambient light issensed by the light sensor, (vi) when an object with user specifiedcolour is sensed by the colour sensor device, (vii) when variation inambient temperature is sensed by a temperature sensor, (viii) whenvariation in ambient humidity is sensed by humidity sensor, (ix) whenpressure crossing a pre-configured threshold is sensed by pressuresensor, (x) when an object is sensed at a pre-configured distance by theultrasonic or infrared distance sensor; and the like.

The said action performed in response to the events/triggers comprises,without limitations: (i) sending command/s to electronic devices orvirtual objects or cloud based services, (ii) performing an action/s inresponse to the trigger or command.

The said Visual Programming Interface comprises any one selected fromthe group consisting of Block Based visual programming interface, (iv)Data Flow Based visual programming interface or Flow Chart Based visualprogramming interface and the like.

The said Interpreted Programming language comprises any one selectedfrom the group consisting of a Javascript, Python, Lua, Typescript, Logoand the like.

The said communication protocol consists of any one selected from thegroup consisting of Wireless Communication with wireless sensors andactuators, Wired Communication with wired sensors and actuators orLocal/Remote Application Programming interface Communication withvirtual objects and Cloud Services.

The said programming and logic constructs comprise any one selected fromthe group consisting of conditional statements like if-then-else, loops,variables, mathematical operators likeaddition-subtraction-multiplication, comparison operators like greaterthan or less than, logical operators such as “AND” or “OR” etc, arraysand lists, numbers, strings, Booleans and other standard programming andlogic constructs and the like.

The said action in response to the trigger or command, withoutlimitation, comprises such as: (i) turning LED on or off in a lightdevice, (ii) turning motor clockwise or anticlockwise for fixed numberof rotations or specified amount of time in motor device, (iii)generating notification in mobile device, (iv) rendering text or imageson display device, (v) playing musical notes or songs or user recordedvoice messages on the speaker device, (vi) sending email with userspecified content to user specified email address, (vii) animating animage object on the screen of a mobile or tablet computer or a laptop ora desktop computer, (viii) posting a message on a social networking sitewith user specified content from user specified social networking handleor account.

The said wireless communication protocol is selected from the groupconsisting of Bluetooth Low energy or Bluetooth or WiFi or ZigBee andthe like. The said wired communication protocol is selected from thegroup consisting of Ethernet, I2C, SPI, UART and the like.

The Local/Remote Application Programming Interface Communicationprotocol is selected from the group consisting of Cloud ServicesApplication Programming interface, Local Software ApplicationProgramming interface and the like.

The Wireless Sensors and Actuators are selected from the groupconsisting of LED, Proximity sensor, Motion sensor, Motor, Screen,speaker, distance sensor, colour sensor, buzzer, button, knob, soundsensor, light sensor, touch sensor, pressure sensor, temperature sensor,moisture sensor, weight sensor, gas sensor, and infrared emitter, andthe like.

The Wired Sensors and Actuators are selected from the group consistingof LED, Proximity sensor, Motion sensor, Motor, Screen, speaker,distance sensor, colour sensor, buzzer, button, knob, sound sensor,light sensor, touch sensor, pressure sensor, temperature sensor,moisture sensor, weight sensor, gas sensor, and infrared emitter, andthe like.

The Virtual Objects are selected form the group consisting of animatedimage, Graphical User Interface components such as button, label,slider, image, chart, notification, and the like.

The Cloud or internet based services are selected from a groupconsisting of Email service, Social network service, text messagingservice and data storage service and the like.

This invention is also embodied in a system of an orchestration enginefor providing a technical output across two or more than twoprogrammable objects selected from the group consisting of electronicdevices, virtual objects and cloud services in response to userspecified logic.

This invention is also embodied as a method of using an OrchestrationEngine for obtaining a technical output across two or more than twoprogrammable objects selected from the group consisting of electronicdevices, virtual objects and cloud services in response to a computerprogram based on user specified logic.

The said system and method of the Orchestration Engine further comprisesthe technical output and an input as provided using the VirtualProgramming Interface, Interpreted Programming Language and are based onthe manner, and by using the communication protocol and electronicdevices; providing event/s or trigger/s generated by one or moreelectronic devices and the action performed, provides action in responseto the trigger or command as described above.

The said method of the Orchestration Engine is capable of supportingextensibility in order to expand support for similar common interactionmethods to newer electronic devices via a plug-in framework byspecifying the communication protocol of the new element and itscapabilities in a descriptive way via a markup language, including JSON,YAML, XML and other languages of similar category.

The Orchestration Engine by itself and as provided in the system and themethod of its use is provided along with a library of drag and dropVisual Programming Language steps required for providing executablecomputer program steps for specifying a user specified logic by acomputer language illiterate person.

BRIEF DESCRIPTION OF THE DRAWINGS

Understanding that these drawings depict only typical embodiments of theinvention and are not therefore to be considered limiting of its scope,the invention will be described and explained with additionalspecificity and detail through the use of the accompanying drawings inwhich:

FIG. 1 provides a schematic diagram that represents the operation of theinvention.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 provides a schematic diagram that represents the operation of theinvention.

In one embodiment this invention comprises a system of wirelesselectronic devices, wired electronic devices, virtual software definedobjects and cloud based services used as building blocks forconstruction of variety of physical computing experiments and prototypesthrough user defined logic captured in the form of visual programminglanguage or an interpreted computer programming language that generatestechnical effect in accordance to user specified logic orchestrated by aportable Orchestration Engine which is implemented as software onvariety of computing platforms like, but not limited to, mobile platformlike tablets or mobiles, a server computer in cloud over the internet,or on the wireless electronic devices in the system. Furthermore, thesaid logic can be executed in multiple modes of operation for example,without limitation, a system of generating technical effect as directedby user specified logic among wireless electronic devices, Thisinvention also comprises a system of generating technical effect asdirected by user specified logic among virtual objects, a system ofgenerating technical effect as directed by user specified logic among acombination of wireless electronic devices and virtual representation ofthese devices; a system of generating technical effect as directed byuser specified logic among a combination of wireless electronic devicesand virtual objects, wired electronic devices and cloud based services.The system is inherently reconfigurable to support various combinationsincluding plurality of same or different wireless electronic devices asphysical computing building blocks and corresponding logic buildingblocks supported in the visual programming language or interpretedtextual programming editor and other computational building blocks forthe construction of logic, for example, without limitation, repeatingstatements e.g. loops, conditional statements e.g. if-then-else, storageof data e.g. variables and arrays and other standard computerprogramming constructs.

This invention comprises a system/method/process/software-enabled devicefor transforming a logical sequence of instructions provided to acomputer as a set of written instructions in a visual programminglanguage or an interpreted computer programming language into a computerreadable software. In a further aspect of this invention, the saidsystem is capable of generating a technical effect by executing thelogic specified by the user by communicating and coordinating withvarious electronic sensors, actuators, virtual objects and cloud basedservices used in the user specified logic using wireless or wiredcommunication channels, local or remote Application ProgrammingInterfaces.

The user can specify execution sequence based on triggers and actionsprovided by plurality of electronic devices, virtual objects, cloudbased services. The electronic devices include, without limitation,light sensors, proximity sensors, buttons and actuators like screen,motor, light, speaker and the like. Virtual objects include, withoutlimitation, an animated image on screen of a mobile device, a touchenabled button or slider on screen of a mobile device or desktopplatform, a virtual representation of the sensor device in form of asimulator or mock object and the like. Users can also specify triggersand actions from other software components, such as, without limitation,services running on the cloud like social networks, database services,email services and the like.

In one aspect of this invention, the logic desired by the user iscaptured using software tool which provides a visual programminginterface. The said software tool can be installed or used from varietyof platforms. This includes, without limitation, mobile or tabletdevices, desktop or laptop computers, web browsers and the like. Oncethe logic is specified, without limitation, such as by feeding into thecomputer the instruction that “when the button on button module ispressed, turn the light on light module on” or “when proximity sensorsenses presence of an object, send an email to my account”, or “when Itouch the button icon on tablet screen, play sound of a drum on thespeaker device”, this invention enables the execution of such a logictriggered via the software tool. The execution of this logic is handledby an Orchestration Engine which is a software program residing on amobile device, or running on a server in cloud or on residing one of theelectronic devices in the system.

In addition to supporting workflow orchestration/co-ordination fortriggers and actions supplied by electronic devices, the system alsosupports coordinated operation of triggers and actions on virtualobjects such as, without limitation, animated images on a mobile ortablet or laptop or desktop computer device e.g. when someone touches ananimated image on the mobile device touch screen, the electronic motormodule should start rotating in clockwise direction. This way, virtualand electronic devices are linked to each other via user specifiedlogic.

In one aspect of the invention, a visual programming interface, whichcan be operated even by a computer programming language illiterateperson, is used as a tool to provide the logical sequence ofinstructions, which gets converted in the computer into a computerprogram written using a standard computer programming language such asC, Java etc. Providing a visual programming interface instead ofstandard computer programming interface allows even the computerprogramming language illiterate individual user to quickly understandhow to write the desired logic in the form of instructions that thecomputer can understand. Examples of visual programming interface areMIT's Scratch programming language (https://scratch.mit.edu/) orCode.org (www.code.org).

Definition of Orchestration Engine: The term “Orchestration Engine”which is also called as “central execution engine” as used in thisspecification, comprises a software component that enables execution ofthe logical sequence of instructions provided to the computer throughvisual programming interface or through Interpreted Programming languageby parsing the specified logic and communicating to other components ofthe system. The other components used in this invention include, withoutlimitation, the electronic devices such as button, display, motor, andservices running on the cloud such as social networks, email servers andthe like, virtual objects such as animated images, simulatedrepresentation of electronic devices, Graphical User Interfacecomponents like label, image, button etc.

While the visual programming language simplifies the process for theuser to specify the desired logical sequence of instructions, theOrchestration Engine provides the ability to execute those instructionsby parsing the specified logic and communicating to other componentssuch as the electronic devices, services running on the cloud, etc. Therequirement for being able to execute the logic is 2 fold

-   -   1. The software component needs to be able to parse and        understand the user specified logic,    -   2. The software component needs to be in a position to        communicate with all different devices and other software        components which are used in the user specified logic.

There are multiple places where the component of the system can belocated as a system component. This includes, without limitation, itsinstallation/insertion in mobile devices including, without limitation,smart phone or tablet, embedded computing devices such as smart-watchesor electronic sensor components, cloud based software, a desktop/laptopcomputer and the like.

The Orchestration Engine is in a position to communicate with allelectronic devices used in the user specified logic via wireless orwired connectivity and with virtual objects and cloud services via localor remote Application Programming Interfaces. Every sensor component,input device, virtual object or cloud service communicates any event itsees to the Orchestration Engine. For example, every time button isclicked, button module reports it or every time some object comes nearproximity sensor, the sensor communicates it to the OrchestrationEngine. The Orchestration Engine, upon receipt of any event checks thelogic specified by the user to see if any action needs to be taken onthe received event. If so, the Orchestration Engine proceeds tocommunicate the necessary actions to be taken to electronic devices,virtual objects or cloud services as instructed by the user specifiedlogic.

As the logic provided by user starts getting more complex and multipleelectronic devices and virtual components are used, it becomes difficultto identify mistakes in the logic when the logic is being executed. Thisis because the elements can communicate much faster and execute theworkflows faster than user can observe. In a further embodiment of thisinvention, to help the user debug incorrect behavior, the methodprovides a way to set the speed of the Orchestration Engine. TheOrchestration Engine then waits for user specified time between everystep in the logic allowing user to analyze the sequence of event at ahuman perceivable speed.

In a still further aspect, the method of this invention is also capableof analyzing the logic specified by the user, compare it against adatabase of logic specified by other users and come up with suggestionsand recommendations to the user about possible improvements to thelogic. The platform can also analyze the complexity of the specifiedlogic using machine learning algorithms run on logic specified bymultiple users. The data is collected in the cloud and is processed tocome up with these statistics.

The wireless electronic devices in the system are capable of detectingpresence of other wireless electronic devices in their vicinity usingbroadcast advertising on the wireless channel and by measuring thereceived signal strength of the wireless broadcasts received fromneighboring devices. This enables the user to specify actions based onevents such as, for example: “when button comes near the light emittingbulb, turn the bulb on”.

The electronic devices are also equipped with an ability to sensecapacitive touch. This is done by using capacitive touch sensitivematerial in manufacturing of the outer shell of the electronic devices.This capacitive sensing capability is similar to the touch-screencapability provided by mobile devices which allows them to sense humantouch. Whenever the electronic devices come in contact with human touch,it is detected by the capacitive sensor in the electronic device whichthen can react to the user action in form of changing the colour ofstatus indicator lights etc. The electronic device also reports this tothe software tool running on the mobile device. The tool then, via theapplication on a mobile or tablet or laptop or desktop computer canindicate which element the user has picked up.

In one aspect of this invention, in order to help the user understandhow to interact with various electronic devices such as a motion sensoror a proximity sensor, the software tool running on a mobile device witha touch-screen provides necessary information in the form of help textand images when an electronic device is kept on the mobile devicescreen, the tool can identify which electronic device user is using.This is achieved by applying a coating of capacitive material in uniqueshape on the enclosure of each electronic device during itsmanufacturing. The mapping between which capacitive shape (i.e.capacitive signature) corresponds to which electronic device ispre-decided and saved in cloud. The software component of the systemrunning on mobile device, using the touch screen interface provided bythe underlying operating system reads the capacitive signature of thedevice kept on the screen. It then refers to the database in cloud toidentify which device is kept on the screen and then displaysappropriate help to the user. For example, when an user places a lightsensor on the designated area on the screen of the mobile device, theapplication will identify it as Light Sensor device based on thecomparison between the capacitive material pattern on the outer shell ofthe said device and a database mapping capacitive patterns andcorresponding devices. The capacitive material pattern can be sensedusing the touch screen capabilities of the operating system and show inhelp and hints about how Light Sensor works and how one can interactwith Light Sensor via a video clip or slide show and other multimediacontent.

The method of this invention also supports extensibility in order toexpand support for the same common interaction methods to newerelectronic devices via a plug-in framework. The plug-in framework allowsthe platform to start supporting newer electronic devices, cloudservices or virtual objects referred to as an “element” by specifyingthe communication protocol of the new element and its capabilities in adescriptive way via a markup language. Markup languages, withoutlimitation include JSON, YAML, XML and the like. No additional computerprogramming is required in the system in order to extend it to supportnewer electronic devices, cloud services or virtual objects. Forexample, adding support for a third party robotics system requires atext file describing the capabilities of the third party robotics systemand communication protocol using a markup language to be added into theplug-in framework. The plug-in framework will parse the file, analyzethe contents and make changes to the user interface to provide visualprogramming blocks as well as interpreted language functions to interactwith the newly added robotic system. The system will now allow users tosend commands to the third party robotics system. Integrating such athird party robotics system does not require any application changes.

The method of this invention provides several advantages. Some of whichinclude:

-   -   1. Teaching computer science and technology to school children.    -   2. Providing quick prototyping capabilities for new products        e.g. validating an industrial automation workflow using smaller        sensor and actuator devices to figure out possible problems        before implementing large scale systems    -   3. Allowing people with non-technology backgrounds and no        electronics and computer programming skills to also start using        customizable technological workflows within their fields.        Examples include, without limitation, an architect can use the        sensors to simulate and demonstrate how lighting automation will        work in a proposed building or a product designer can use the        system to simulate and demonstrate how a proposed medical device        design will operate.

Following are some examples on to illustrate working of this invention.These examples, by no means, limit the scope of this invention. A personskilled in the art would readily understand that variation of theseexamples. Several other and equivalent examples are possible asapplication of this invention; all such obvious variations andequivalents are construed to be within the scope of this invention.

EXAMPLES Example 1

Application: An architect demonstrating proposed security system for anew Home design project where the model of the house is made to scaleusing cardboard or similar material and an alert is sent via email tothe owner of the home

Objective: When the security system is enabled and an intruder entersthe house, alarm should to go off automatically to alert everyone. Thehome owner is alerted by sending an email alert.

Programmable objects used: Button module, Proximity sensor, Buzzermodule

Arrangement of electronic devices/hardware: The security system wasconstructed by keeping a proximity sensor just behind the entrance doorof the model house and a buzzer module in another room of the modelhouse. A button module placed near the buzzer was used to enable ordisable the system.

Instructions: A logical sequence of instructions was created in theapplication to represent the desired behavior of the system.

The instructions were as follows:

-   -   1. when button is clicked, enable the system    -   2. when object comes near proximity,        -   a. if: system is enabled            -   i. start the buzzer            -   ii. send email to home owner        -   b. else: do nothing    -   3. when button is double clicked, disable the system

For each of the above pre-constructed instruction in visual programminglanguage, appropriate algorithm to actuate relevant devices was fed intothe relevant electronic devices.

Example 2

Application: A child demonstrating an automatic street lighting systemas part of the ‘smart city’ theme science exhibition in their school.

Objective: The street light system should be activated when it becomesdark and there is movement in the vicinity of the street lights.

Programmable objects used: Proximity sensor, 3 Light modules, Lightsensor

Arrangement of electronic components/hardware: The light modules wereplaced at fixed intervals attached to candy sticks erected to depictstreet lights along the street constructed using cardboard cutouts andpainted using coloring materials. A light sensor and proximity sensorwere placed near the first street light.

Instructions: A logical sequence of instructions was created in theapplication to represent the desired behavior of the system.

The instructions were as follows:

-   -   1. when the surrounding light changes from bright to dark,        activate the system    -   2. when object comes near proximity,        -   a. if: system is activated            -   i. turn lights on in white color            -   ii. turn lights off after fixed time        -   b. else: do nothing    -   3. when the surrounding light changes from dark to bright,        deactivate the system

For each of the above pre-constructed instruction in visual programminglanguage, appropriate algorithm to actuate relevant devices was fed intothe relevant electronic devices.

Example 3

Application: A group of girls in an after-school robotics workshopcreating a hand controlled vehicle project that includes an userinterface to control the movements of the vehicle.

Objective: To operate the vehicle using the motion sensor held in thehand

Programmable objects used: Motion sensor, Motors, Graphical userinterface (GUI) button components.

Arrangement of electronic components/hardware: Two motors (A and B) werejoined together using elastic bands and wheels were attached to themotor shaft to create a drive platform. The motion sensor was held inthe hand such that it could be tilted front or back. Two GUI buttoncomponents were used to create a user interface that controlled left andright turn of the vehicle.

Instructions: A logical sequence of instructions was created in theapplication to represent the desired behavior of the system.

The instructions were as follows.

-   -   1. When motion sensor detects forward tilt, start rotating motor        A in clockwise direction and start rotating motor B in        anticlockwise direction.    -   2. When motion sensor detects backward tilt, stop rotating both        motors.    -   3. When the left button is clicked on the user created GUI, stop        motor A, rotate motor B in anticlockwise direction for 1        rotation and then start both the motors to move the drive        platform forward.    -   4. When the right button is clicked on the user created GUI,        stop motor B, rotate motor A in clockwise direction for 1        rotation and then start both the motors to move the drive        platform forward.

For each of the above pre-constructed instruction in visual programminglanguage, appropriate algorithm to actuate relevant devices was fed intothe relevant electronic devices.

Example 4

Application: A designer in a design studio creating an auto adjustingpulley system model which is part of the new machine designed by her forthe client.

Objective: To operate a pulley system that can be activated usingbuttons but stops automatically when it approaches ground or the pulleypivot point.

Programmable objects used: 2 Proximity sensors, Button module, Motor,Pulley (accessory), String (accessory), container to attach to string inwhich objects can be placed.

Arrangement of electronic components/hardware: One proximity sensor wasattached just below the pulley. Another proximity sensor was attached tothe bottom of the container tied to a string. The string was setuparound the pulley. Pulley was attached to the motor shaft. Motor wasplaced on some height.

Instructions: A logical sequence of instructions was created in theapplication to represent the desired behavior of the system.

The instructions were as follows.

-   -   1. When button is clicked, start motor in clockwise direction so        that the object attached to pulley starts descending.    -   2. When button is double-clicked, start motor in anticlockwise        direction so that the object attached to pulley starts        ascending.    -   3. When proximity sensor attached to the container senses        something near it, stop the motor.    -   4. When proximity sensor attached below the pulley senses        something near it, stop the motor.

For each of the above pre-constructed instruction in visual programminglanguage, appropriate algorithm to actuate relevant devices was fed intothe relevant electronic devices.

Example 5

Application: At home, a child creating a working model of an automaticboom barrier for train crossing which he has seen recently during afamily road trip.

Objective: To operate a boom barrier as per the train passing by thecrossing.

Programmable objects used: 2 Proximity sensors, Motor, stick to act asboom barrier (accessory)

Arrangement of electronic components/hardware: A stick acting as boombarrier was attached to the motor. The motor was placed in front of therailway crossing in the train set. Proximity sensor A was placed on theside of the boom barrier from where train approaches. Proximity sensor Bwas placed on the side of the boom barrier from where the train goesaway.

Instructions: A logical sequence of instructions was created in theapplication to represent the desired behavior of the system.

The instructions were as follows.

-   -   1. When proximity sensor A senses something near it, turn the        motor by in anticlockwise direction by quarter rotation so that        the barrier closes.    -   2. When proximity sensor B senses something near it, turn the        motor in clockwise direction by quarter rotation so that the        barrier opens.

For each of the above pre-constructed instruction in visual programminglanguage, appropriate algorithm to actuate relevant devices was fed intothe relevant electronic devices.

Example 6

Application: A teacher in a computational thinking class creating anautomated traffic light system as a practical application of thealgorithm the class has learnt during the session.

Objective: To control an automated traffic light system at a Y shapedjunction.

Programmable objects used: 3 Light modules

Arrangement of electronic components/hardware: 3 light modules wereplaced on each stem of a Y shaped junction to act as guiding trafficlight for vehicles coming from that stem.

Instructions: A logical sequence of instructions was created in theapplication to represent the desired behavior of the system.

The instructions were as follows.

-   -   1. In a continuous loop,    -   2. Turn light 1 green, turn light 2 and 3 red.    -   3. Wait for 30 seconds    -   4. Turn light 1 yellow.    -   5. Wait for 5 seconds.    -   6. Turn light 2 green, turn light 1 and 3 red.    -   7. Wait for 30 seconds    -   8. Turn light 2 yellow.    -   9. Wait for 5 seconds.    -   10. Turn light 3 green, turn light 1 and 2 red.    -   11. Wait for 30 seconds    -   12. Turn light 3 yellow.    -   13. Wait for 5 seconds.

For each of the above pre-constructed instruction in visual programminglanguage, appropriate algorithm to actuate relevant devices was fed intothe relevant electronic devices.

Example 7

Application: Students in a computer lab in an under-graduate coursecreating a fan regulator system to understand the basic codingconstructs of loops, variables and conditional statements.

Objective: To control a fan using a knob as a regulator

Programmable objects used: Knob module, Motor module, a fan to attach tothe motor (accessory).

Arrangement of electronic components/hardware: A fan was attached to themotor shaft.

Instructions: A logical sequence of instructions was created in theapplication to represent the desired behavior of the system.

The instructions were as follows:

-   -   1. In a continuous loop read the knob value and store the value        in a variable    -   2. If the knob value is zero, stop the motor    -   3. If the knob value is between 0 and 33 (inclusive), start        rotating the motor clockwise at slow speed.    -   4. If the knob value is between 34 and 66 (inclusive), start        rotating the motor clockwise at medium speed.    -   5. If the knob value is between 67 and 99 (inclusive), start        rotating the motor clockwise at fast speed.

For each of the above pre-constructed instruction in visual programminglanguage, appropriate algorithm to actuate relevant devices was fed intothe relevant electronic devices.

Example 8

Application: A computer science teacher demonstrating the use ofconditional statements to the class by using a button as toggling lightswitch.

Objective: To use a button as a state toggling light switch.

Programmable objects used: Button module, Light module

Arrangement of electronic components/hardware: No special arrangementsneeded.

Instructions: A logical sequence of instructions was created in theapplication to represent the desired behavior of the system.

The instructions were as follows:

-   -   1. When the program starts, create a state variable and set it        to “off”.    -   2. When the button is clicked, check the variable value    -   3. If the variable is “off”, turn the light on and set the        variable to “on”.    -   4. If the variable is “on”, turn the light off and set the        variable to “off”.

For each of the above pre-constructed instruction in visual programminglanguage, appropriate algorithm to actuate relevant devices was fed intothe relevant electronic devices.

What is claimed:
 1. An Orchestration Engine for providing a technicaloutput across two or more than two programmable objects selected fromthe group consisting of electronic devices, virtual objects and cloudservices in response to user specified logic.
 2. The OrchestrationEngine of claim 1, wherein: a. the orchestration engine is deployed on amobile computer, a tablet computer, a laptop computer, a desktopcomputer, a wired or wireless electronic device, or a server computer,b. the technical output comprises one or more actions performed inresponse to an event or a trigger generated by the programmable objectsand based on a combination of different programming and logicconstructs, c. the orchestration engine receives the user specifiedlogic through: i. a Visual Programming Interface, or ii. an InterpretedProgramming language; and the Orchestration Engine executes the userspecified logic using a communication protocol; d. the orchestrationengine has one or more of the following features: i. the orchestrationengine provides the user with means for speed setting and waits for auser specified time between every step in the user specified logic tothereby allow the user to analyze a sequence of events at a humanperceivable speed, ii. the orchestration engine analyzes and comparesthe user specified logic against a database of logic specified by otherusers to thereby generate and present suggestions and recommendations tothe user about possible improvements to the user specified logic, e. theprogrammable objects to which the technical output is provided have oneor more of the following features: i. a capability of detecting presenceof other programmable objects using broadcast advertising on a wirelesschannel and by measuring a received signal strength of other wirelessbroadcasts received from the other programmable objects, or ii. acapability to sense capacitive touch, by using capacitive touchsensitive material in manufacturing of an outer shell of theprogrammable objects, and in response to a sensed capacitive touch,notify the Orchestration Engine to thereby enable the OrchestrationEngine to display, via a graphical user interface which programmableobject the user has touched and to display help to the user related tothe touched programmable object.
 3. The Orchestration Engine of claim 2wherein: a. the event or the trigger comprises any one of: i. adetection of a presence of another object in vicinity of a proximitysensor device, ii. a detection of acceleration or movement in anydirection by a motion sensor device, iii. a detection of receipt of anemail with user specified content at a user specified account, iv. adetection of a selection of an image displayed on a screen, v. adetection of a variation in ambient light by a light sensor, vi. adetection of an object with user specified color by a color sensordevice, vii. a detection of a variation in ambient temperature by atemperature sensor, viii. a detection of a variation in ambient humidityby a humidity sensor, ix. a detection of a pressure crossing apre-configured threshold by a pressure sensor, x. a detection of anobject within a pre-configured distance by an ultrasonic or infrareddistance sensor; b. the one or more actions comprise: i. sending one ormore commands to the programmable objects, c. the Visual ProgrammingInterface comprises any one selected from a group consisting of: a BlockBased visual programming interface, a Data Flow Based visual programminginterface or a Flow Chart Based visual programming interface, d. theInterpreted Programming language comprises any one selected from a groupconsisting of: Javascript, Python, Lua, Typescript, or Logo, e. thecommunication protocol consists of any one selected from a groupconsisting of: Wireless Communication with wireless sensors andactuators, Wired Communication with wired sensors and actuators or aLocal or Remote Application Programming Interface Communication withvirtual objects and Cloud Services, f. the programming and logicconstructs comprises any one selected from a group consisting of:conditional statements, mathematical operators, comparison operators,logical operators, arrays, lists, numbers, strings and Booleans.
 4. TheOrchestration Engine of claim 3 wherein the one or more actions compriseone or more of: a. turning an LED on or off in a light device, b.turning a motor clockwise or counterclockwise for a fixed number ofrotations or a specified amount of time in a motor device, c. generatinga notification in a mobile device, d. rendering text or images on adisplay device, e. playing musical notes or songs or user recorded voicemessages on a speaker device, f. sending an email with user specifiedcontent to a user specified email address, g. animating an image objecton a screen of a mobile device, a tablet, a laptop or a desktopcomputer, h. posting a message on a social networking site with userspecified content from a user specified social networking handle oraccount.
 5. The Orchestration Engine of claim 2 wherein: a. the wirelesscommunication protocol is selected from a group consisting of: BluetoothLow energy, Bluetooth, WiFi or ZigBee, b. the wired communicationprotocol is selected from a group consisting of: Ethernet, I2C, SPI orUART, c. the Local or Remote Application Programming InterfaceCommunication is selected from a group consisting of: a Cloud ServicesApplication Programming interface or a Local Software ApplicationProgramming interface, d. the Wireless Sensors and Actuators areselected from a group consisting of: and LED, a Proximity sensor, aMotion sensor, a Motor, a Screen, a speaker, a distance sensor, a colorsensor, a buzzer, a button, a knob, a sound sensor, a light sensor, atouch sensor, a pressure sensor, a temperature sensor, a moisturesensor, a weight sensor, a gas sensor, and an infrared emitter, e. theWired Sensors and Actuators are selected from a group consisting of: anLED, a Proximity sensor, a Motion sensor, a Motor, a Screen, a speaker,a distance sensor, a color sensor, a buzzer, a button, a knob, a soundsensor, a light sensor, a touch sensor, a pressure sensor, a temperaturesensor, a moisture sensor, a weight sensor, a gas sensor, and aninfrared emitter, f. the Virtual Objects are selected from a groupconsisting of: an animated image, a button, a label, a slider, an image,a chart and a notification, g. the cloud services are selected from agroup consisting of: an Email service, a Social network service, a textmessaging service and a data storage service.
 6. The OrchestrationEngine of claim 1 wherein the Orchestration Engine is provided alongwith a library of drag and drop Visual Programming Language steps forproviding executable computer program steps to enable a computerlanguage illiterate person to provide the user specified logic.
 7. Asystem to provide a technical output across two or more than twoprogrammable objects selected from the group consisting of electronicdevices, virtual objects and cloud services in response to userspecified logic, the system comprising: a computer including one or moreprocessors; a user interface running on the computer to enable a user tospecify logic using a Visual Programming Interface or using anInterpreted Programming Language; and an orchestration engine running onthe computer to interpret, execute and monitor the user specified logicand to provide technical output across two or more than two programmableobjects selected from a group consisting of electronic devices, virtualobjects and cloud services and to communicate through WirelessCommunication with Wireless Sensors and Actuators, or through WiredCommunication with Wired Sensors and Actuators, or through local orremote API communication with Virtual Objects and Cloud Services, inaccordance with the user specified logic.
 8. The system of claim 7wherein: the computer is any one or more selected from a groupconsisting of: a mobile computer, a tablet computer, a laptop computer,a desktop computer, a wired or wireless electronic device or a servercomputer connected via the internet; the user interface is accessed viaa touch screen, a keyboard or a mouse; the Visual Programming Interfaceis any one or more selected from a group consisting of: a Block BasedProgramming Interface, a Data Flow Based Programming Interface or a FlowChart based Programming Interface; the Interpreted Programming Languageis any one or more selected from a group consisting of: Javascript,Python, Lua, Typescipt or Logo; the user specified logic comprises:programming and logic constructs selected from a group consisting of:if-then-else statements, loops variables, mathematical operators,addition-subtraction-multiplication statements, comparison operators,greater than operators or less than operators, logical AND or ORoperators, and data types selected from a group consisting of: arrays,lists, numbers, strings or Booleans; the orchestration engine providesthe user with a speed setting and waits for a user specified timebetween every step in the user specified logic allowing the user toanalyze a sequence of events at a human perceivable speed; theorchestration engine analyzes and compares the user specified logicagainst a database of logic specified by other users and generates andpresents suggestions and recommendations to the user about possibleimprovements to the user specified logic; the technical output comprisesone or more actions performed in response to an event or a triggergenerated by one or more electronic devices, virtual objects or cloudbased services in accordance with the user specified logic; the wirelesscommunication is performed using a protocol selected from a groupconsisting of: Bluetooth Low energy, Bluetooth, WiFi or ZigBee; thewired communication is performed using a protocol selected from a groupconsisting of: Ethernet, I2C, SPI or UART; the local or remote APICommunication is performed using an interface selected from a groupconsisting of: a Cloud Services Application Programming interface or aLocal Software Application Programming interface, the Wireless Sensorsand Actuators are selected from a group consisting of: an LED, aProximity sensor, a Motion sensor, a Motor, a Screen, a speaker, adistance sensor, a color sensor, a buzzer, a button, a knob, a soundsensor, a light sensor, a touch sensor, a pressure sensor, a temperaturesensor, a moisture sensor, a weight sensor, a gas sensor, and aninfrared emitter; the Wired Sensors and Actuators are selected from agroup consisting of: an LED, a Proximity sensor, a Motion sensor, aMotor, a Screen, a speaker, a distance sensor, a color sensor, a buzzer,a button, a knob, a sound sensor, a light sensor, a touch sensor, apressure sensor, a temperature sensor, a moisture sensor, a weightsensor, a gas sensor, and an infrared emitter; the Virtual Objects areselected from a group consisting of: an animated image, a button, alabel, a slider, an image, a chart or a notification; and the Cloudservices are selected from a group consisting of: an Email service, aSocial network service, a text messaging service and a data storageservice.
 9. The system of claim 7 further comprising: a library of dragand drop Visual Programming Language steps for providing executablecomputer program steps to enable a language illiterate person to providethe user specified logic.
 10. A method for providing a technical outputacross two or more than two programmable objects selected from the groupconsisting of electronic devices, virtual objects and cloud services inresponse to user specified logic comprising: providing a computerincluding one or more processors; specifying the user specified logicusing a Visual Programming Interface or using an Interpreted ProgrammingLanguage via a user interface running on the computer; storing the userspecified logic on the computer; interpreting, executing and monitoringthe user specified logic via an orchestration engine running on thecomputer; and providing technical output across two or more than twoprogrammable objects selected from a group consisting of: electronicdevices, virtual objects and cloud services, the technical output beingprovided by communicating through Wireless Communication with WirelessSensors and Actuators, through Wired Communication with Wired Sensorsand Actuators, or through Local or Remote API communication with CloudServices and Virtual Objects in response to the user specified logicprovided to the computer.
 11. The method of claim 11, furthercomprising: providing a plug-in framework by specifying communicationprotocol capabilities of a new element via a markup language, the markuplanguage including one or more of: JSON, YAML or XML to thereby supportextensibility for newer electronic devices.
 12. The method of claim 11wherein: the computer is any one or more selected from a groupconsisting of: a mobile computer, a tablet computer, a laptop computer,a desktop computer, a wired or wireless electronic device or a servercomputer connected via the internet; the user interface is accessed viaa touch screen, a keyboard or a mouse; the Visual Programming Interfaceis any one or more selected from a group consisting of: a Block BasedProgramming Interface, a Data Flow Based Programming Interface or a FlowChart based Programming Interface; the Interpreted Programming Languageis any one or more selected from a group consisting of: Javascript,Python, Lua, Typescipt or Logo; the user specified logic comprises:programming and logic constructs selected from a group consisting of:if-then-else statements, loops variables, mathematical operators,addition-subtraction-multiplication statements, comparison operators,greater than operators or less than operators, logical AND or ORoperators, and data types selected from a group consisting of: arrays,lists, numbers, strings or Booleans; the orchestration engine providesthe user with a speed setting and waits for a user specified timebetween every step in the user specified logic allowing the user toanalyze a sequence of events at a human perceivable speed; theorchestration engine analyzes and compares the user specified logicagainst a database of logic specified by other users and generates andpresents suggestions and recommendations to the user about possibleimprovements to the user specified logic; the technical output comprisesone or more actions performed in response to an event or a triggergenerated by one or more electronic devices, virtual objects or cloudbased services in accordance with the user specified logic; the wirelesscommunication is performed using a protocol selected from a groupconsisting of: Bluetooth Low energy, Bluetooth, WiFi or ZigBee; thewired communication is performed using a protocol selected from a groupconsisting of: Ethernet, I2C, SPI or UART; the local or remote APICommunication is performed using an interface selected from a groupconsisting of: a Cloud Services Application Programming interface or aLocal Software Application Programming interface, the Wireless Sensorsand Actuators are selected from a group consisting of: an LED, aProximity sensor, a Motion sensor, a Motor, a Screen, a speaker, adistance sensor, a color sensor, a buzzer, a button, a knob, a soundsensor, a light sensor, a touch sensor, a pressure sensor, a temperaturesensor, a moisture sensor, a weight sensor, a gas sensor, and aninfrared emitter; the Wired Sensors and Actuators are selected from agroup consisting of: an LED, a Proximity sensor, a Motion sensor, aMotor, a Screen, a speaker, a distance sensor, a color sensor, a buzzer,a button, a knob, a sound sensor, a light sensor, a touch sensor, apressure sensor, a temperature sensor, a moisture sensor, a weightsensor, a gas sensor, and an infrared emitter; the Virtual Objects areselected from a group consisting of: an animated image, a button, alabel, a slider, an image, a chart or a notification; and the Cloudservices are selected from a group consisting of: an Email service, aSocial network service, a text messaging service and a data storageservice.
 13. The method of claim 11, further comprising: providing alibrary of drag and drop Visual Programming Language steps for providingexecutable computer program steps to enable a computer languageilliterate person to provide the user specified logic.